Pruning Fuzzy ARTMAP using the Minimum Description Length Principle in Learning from Clinical Databases

Fuzzy ARTMAP is one of the families of the neural network architectures bused on ART(Adaptive Resonance Theory) in which supervised learning can be curried out. However, it usually tends to create more categories than are actually needed. This often causes the so culled overfitting problem, namely the performunce of the networks in test set is not monotonically increasing with the additional training epochs and cutegory creation, j&-fuzzy ARTMAP. In order to avoid the overfitting problem, Carpenter und Tun [Carpenter and Tan, 19931 proposed a confidence-based pruning method by eliminating those categories that were either less useful or less accurate. This puper proposes yet another alternative pruning method thut is bused on the Minimal Description Length (MDL) principle. The MDL principle can be viewed as a tradeoflbetween theory complexity and data prediction accurucy given the theory. We adopted CumeronJones’ error encoding scheme and Quinlan’s mod@er for theory encoding to estimate the fuzzy ARTMAP theory description length. A greedy search algorithm of the minimum description length to prune the fiizzy ARTMAP categories one by one is proposed. The experiments sh.owed that fuzzy ARTMAY pruned with the MDL principle gave better performance with .fur fewer categories created than the original fuzzy ARTMAP and other machine learning systems on a number of benchmark clinical dutabases such as heart disease, breath cancer and diabetes databases. (Subject Area: Neural Networks; Knowledge Acquisition and Machine Learning)